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A HOMING PROBLEM FOR DIFFUSION PROCESSES WITH 
CONTROL-DEPENDENT VARIANCE 1 

By Mario Lefebvre 
Ecole Polytechnique de Montreal 

Controlled one-dimensional diffusion processes, with infinitesimal 
variance (instead of the infinitesimal mean) depending on the control 
variable, are considered in an interval located on the positive half-line. 
The process is controlled until it reaches either end of the interval. 
The aim is to minimize the expected value of a cost criterion with 
quadratic control costs on the way and a final cost equal to zero 
(resp. a large constant) if the process exits the interval through its 
left (resp. right) end point. Explicit expressions are obtained both 
for the optimal value of the control variable and the value function 
when the infinitesimal parameters of the processes are proportional 
to a power of the state variable. 

1. Introduction. Let x(t) be a one-dimensional controlled diffusion pro- 
cess defined by the stochastic differential equation 

dx(t) = a[x(t),t] dt + b[x(t),t]u(t) dt + {v[x(t),t}} 1/2 dW(t), 

where v(-, •) is positive and W(t) is a standard Brownian motion, and define 

t(x) :=mf{t >0:x(t) £ {di,d 2 } \ x(0) =x£ [di,d 2 ]}. 

Using a result due to Whittle [(1982), page 289], we can show that if the 
uncontrolled process y(t) that corresponds to x(t) is certain to leave the in- 
terval [di,^] j then the value of the control u(t) that minimizes the expected 
value of the cost criterion 

rr(x) 

J(x)= / \q[x{t),i\u 2 {t)dt + K(x(t),t), 
Jo 
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2 M. LEFEBVRE 

where q(-, •) is positive and K(-, •) is a general termination cost function, can 
be obtained from the mathematical expectation 

E[e- K{y{T) ' T)/a \y(0) = x], 

in which a is a positive parameter such that the relation 

v = ab 2 /q 

holds. Actually, the mathematical expectation above is equal to e~ F ( x '' a , 
where F(x) is the value function defined by 

(1) F(x)= inf E\J(x)\. 

U (t),0<t<T(x) 

Whittle has termed this type of problem LQG homing. It can be general- 
ized [see Lefebvre (1989, 1997)] in particular by using a risk-sensitive cost 
criterion instead of J(x) [see Kuhn (1985) and Whittle (1990), page 222]. 

Next, the author [Lefebvre (2001)] modified the problem set up by Whittle 
by considering the controlled process defined by 

dx(t) = b[x(t)]u(t) dt + {v[x{t)]\u(t)\} 1/2 dW{t) 

and the cost criterion 

J(x)= [ T{X aq[x(t)}u 2 (t) + X}dt, 



-2 l 

where q(-) is positive, A is a positive parameter and 

r(x) :=inf{t >0:\x(t)\ =d\x(0) = x€ [-d,d]}. 

He found, under some symmetry assumptions, that the control that mini- 
mizes the expected value of J(x) is given by 

u* = I — — ) when < x < d. 

\q{x)J 

He also gave a probabilistic interpretation to the value function F{x) and, 
finally, he computed explicitly this function in the most important cases, 
for instance, the cases when x(t) with u(t) = 1 is a Wiener process or a 
geometric Brownian motion. 

In the present paper, we assume that the controlled stochastic process X{t) 
obeys the stochastic differential equation 

(2) dX(t) = f[X(t)\ dt + {v[X(t)]\u(t)\} 1 / 2 dW(t), 

where f(x) and v{x) are positive functions for x > d\, and we let 

t{x) := inf {t > : X(t) £ {d 1 ,d 2 } \ X(0) =x£ [di, d 2 ]} 
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with d\ > 0. Therefore, it is the infinitesimal variance of the controlled pro- 
cess X(t) that is control-dependent, rather than its infinitesimal mean, as in 
Whittle [(1982), page 289]. The cost criterion is 

(3) J(x) = [ T[X a q [X(t)]u 2 {t) + \}dt + K{X[t(x)}}. 

Jo 

In J(x), the parameter A can now take any real value and K(-) is defined 
by 



K{X[r(x)}} 



0, ]fX[r(x)] = d 1 , 
K , ifX[ T (x)} = d 2 , 



where Kq is a (large enough) positive constant. Thus, if A is positive, then 
the aim is to make the controlled process X(t) leave the interval (d\,d 2 ) 
as soon as possible and through its left end point d±, whereas when A is 
negative there is a reward given for survival in (d±, d 2 ). If A = 0, time spent 
in the continuation region (d±, d 2 ) is neither directly rewarded nor penalized; 
however, because the function q(-) is assumed to be strictly positive, the 
sooner X(t) exits (di,d 2 ), the better. In all cases, the quadratic control 
costs must of course be taken into account. 

In the next section, the optimal value of the control variable will be com- 
puted. The case when the functions /, v and q are proportional to X n (t), 
where n € {0, 1, 2}, will be treated thoroughly. Particular examples, includ- 
ing the case when X(t) with u(t) = 1 is a Wiener process, will be presented 
in Section 3. Finally, a few concluding remarks will be made in Section 4. 

2. Computation of the optimal control. Let F{x) be the value function 
defined in (1). Assuming that it exists and is twice differentiable, we can 
easily show that it satisfies the dynamic programming equation 

(4) = inf {\q{x)u 2 + A + f{x)F'{x) + \v(x)\u\F" (x)} 

for d\ < x < d 2 , where u := u(0). Moreover, the boundary conditions are 

(5) F(d 1 )=0 and F(d 2 )=K . 

Now, since u(t) only appears in absolute value in the stochastic differ- 
ential equation (2) and squared in the cost criterion (3), the sign of u(t) 
is actually irrelevant. Hence, we can assume without loss of generality that 
u is nonnegative and it then follows at once that the optimal control u* is 
given by 

(6) „. = _^ F « (I)( >„). 
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Remarks. 1. We will have to check below that F"(x) is indeed less 
than or equal to zero. Actually, we cannot have F"[x) = and satisfy both 
boundary conditions in (5). So, F"(x) should in fact be strictly negative. 

2. We have assumed above that the function f(x) is positive if x > d\. 
Notice that if f(x) is negative for x S [di, d?\ and A = 0, then the optimal 
control is trivially given by u* = 0. Indeed, we then obtain that F(x) = 
(for d\ < x < (I2), which is clearly the smallest value that F{x) can take. 
However, when we choose a parameter A different from zero, the case 
f(x) 7^ could be considered. 

3. If the parameter A = 0, then the function F(x) takes its values in the 
interval [0, Kq[. 

4. In some cases, the origin is an inaccessible boundary for the uncontrolled 
process X\(t) obtained by setting u(t) = 1; that is, the origin cannot 
be reached in finite time. This is true, in particular, when X\(t) is a 
geometric Brownian motion defined by the stochastic differential equation 

dX 1 (t)=X 1 (t)dt + \X 1 (t)\dW(t). 

Therefore, it is natural to choose d± strictly positive in such How- 
ever, in other cases d\ can be chosen equal to zero in a very legitimate way. 

Next, substituting the optimal control u* into the dynamic programming 
equation (4), we find that the value function F{x) satisfies the second-order 
nonlinear ordinary differential equation 

(7) = X + f( x )F'(x)-^[F"(x)] 2 . 

Summing up, we may state the following proposition. 

Proposition 2.1. If the value function F(x) exists and is twice differ- 
entiable, then the optimal control u* is given by the formula (6). Moreover, 
the function F(x) can be obtained by solving the ordinary differential equa- 
tion (7), subject to the boundary conditions (5). 

Remark. Note that for the problem set up above to make sense, we 
must have A + f(x)F'(x) > [see (7)]. 

Next, we will solve explicitly the nonlinear ordinary differential equa- 
tion (7) in two particular cases. 

Case 1. Assume first that f(x) = /o, a positive constant, and that 

(8) p&:=h(x) = hox n , 

where /iq is also a positive constant and n S {—2, — 1, . . . , 4}. 
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Remark. The most important cases for applications are the ones when 
f(x) = fox k , v(x) = vox^ and q(x) = qox 1 with j, k, I G {0, 1, 2}. Actually, we 
could also include the case when f(x) = fo/x. Indeed, X(t) with u(t) = 1 
could then be a Bessel process [if v(x) = 1; see Karlin and Taylor (1981), 
pages 175 and 176, for instance]. 

When the formula (8) holds [and f(x) = / ], we may rewrite the ordinary 
differential equation (7) as 

(9) h(x)[F"(x)] 2 = X + f F'(x). 
Differentiating this differential equation, we obtain that 

h'{x)[F"{x)} 2 + 2h{x)F"{x)F"'{x) = f F"{x). 
Hence, because we must have F"(x) < (see above), we may write that 

(10) 2h(x)G'(x) + h'(x)G(x)=f , 

where G(x) :=F"(x). The general solution of (10) is given by 

G(x)=h~ 1 /\x)[f J-^ ) dx + cy 

where c is a constant. It is now easy to obtain an explicit expression in the 
case when h(x) = hox n . We find that 

G ( X ) = ^=x~ n l 2 + ^ ^— - if n + 2 



>ho ho 2 — n 

(11) and 

G{x) = ^ l - + ^ ifn = 2. 
V/to x ho 2x 

Integrating the function G(x) twice, we obtain that the value function F(x) 
is given by 

2c x 2 ~^ f x 3 ~ n 

+ 1 — 77; 7777^ 7 + c l x + c 



/%(2-n)(2-n/2) h (2 - n) 2 (3 - n) 

if 2, 3, 4, 

c _n„ _ -,\ , fo _n. 2 



x(lnx — 1) + — — x(ln x — 2lnx + 2) + c\x + cq 



(12) 



% 4/i 

if n = 2, 

=4x 1 / 2 + ^- \nx + c\x + Co if n = 3, 

/i «o 



mx — — h c\x + cq itn = 4, 



/i n 4/i x 
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where c\ and cq are constants. 

Finally, the constants c, c\ and Co are uniquely determined from the 
boundary conditions (5) and the equation (9). Actually, we find that c 2 = 
A + /o c i- An example will be presented in the next section. 



Case 2. If A = and 

v 2 (x) 



■=9(x) =gox m , 



8q(x)f(x) 

where go is a positive constant and m £ {—4, —3, . . . , 4}, we have 
(13) g(x)[F"(x)] 2 = F'(x). 

Proceeding as above, we find that the function F(x) is given by (12), with 
fo = 1 , n replaced by m and ho by go , and that the constants c, c\ and 
Co are now uniquely determined from the boundary conditions (5) and the 
ordinary differential equation (13). Here, we find that c 2 =c\. As for Case 1, 
an example will be provided in the next section. 

Remarks. 

1. As mentioned above, we must also check that the condition F"(x) < is 
satisfied. 

2. As will be seen in the examples presented in Section 3, there is also a 
restriction on the constant Kq in the definition of the function K(-). 

3. Examples, (a) First, we consider the particular case when f(x) = fo, 
v(x) = vq and q(x) = qo, where fo, vq and qo are all positive constants. 
Then, the controlled process X{i) with u{t) = 1 is a Wiener process with 
infinitesimal parameters fo and vq. We have 

h(x) = h = ^, 
8qo 

so that n = in (8). It follows that [see (12)] 

2/o<?o 3 , c V2q~o 



F(x) = - 9 x H x +c\x + co- 

3t>o vo 

We find that the ordinary differential equation (9) is satisfied if and only if 
we take c 2 = A + foC\ , as noticed above. It follows that 

ey \ 2 /ogo 3 , c \/2go 2 , c 2 — A 
F(x) = x H x H — x + cq. 

3^o fo 
Next, the boundary condition F(d\) = implies that 



F(x) = ^f(, 3 - d\) + ^°(x 2 - dj) + ^(x- dl ), 
3^o v fo 
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and finally, F(d,2) = Kq yields that 

„ c 2 (d 2 -d 1 ) c^rjj-df) 2/ ogo (d|-^) A(d 2 -d!) 

A = 7 1 1 7T2 f ' 

Jo v 3v$ Jo 

Hence, we have 



o 



2(d 2 - di) 

y/Wo{4-d\) 



± 



wo 

\ — « 



f 2(7o (d|-df; 



o 



^ d 2 -di ^ 2/ g (d|-df) A(d 2 -di) ^ 

4 ~e 7 A t 

Jo V 3^ /o 



1/2 



To simplify further, we take fo = Vq = 1 and go = 1/2. We get that 
(d 2 + di) 
C = T~ 

(14) 

b {(<% ~ dj) 2 - 4(d 2 - d!)(l/3(d| - d|) - A(d 2 - di) - if )}V 2 

2(d 2 -di) 

We then deduce that the constant Kq must satisfy the inequality 

(15) K >X(d 2 -d 1 ) + ^(d 2 -d 1 ) 3 . 

Notice that the larger the parameter A is, the larger Kq must be. Conversely, 
if A is (negative and) small enough, any Kq > is admissible. 

To determine whether we must choose the "+" or "— " sign in (14), we 
will use the fact that we must have F"(x) < 0; that is, with /q = vq = 1 and 

r, — 1 

go - 2i 

F"(x) = G(x) =2x + 2c<0 fordi<x<d 2 . 

This implies that the constant c must be smaller than or equal to — d 2 . 
Using (14), we obtain that 

{(d 2 _ d 2)2 _ 4{d2 _ dl){ l /3{d 3 _ d 3) _ X{d2 _ dl) _ Kq)} 1/2 

2(d 2 -di) 

(16) d d 

<^«o). 

Thus, we must choose the "— " sign. Then, we find that (16) implies that 



(d| - d\f - 4(d 2 - d^Udl - of) - A(d 2 - dx) - K ) > (d 2 - d 1 ) A 



K > A(d 2 - di) + |(d 2 - di) 3 . 
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Since this last condition is more restrictive than the one in (15), this is a 
condition that must be fulfilled. 

Summing up, when we choose f(x) = 1, v(x) = 1 and q(x) = i, the optimal 
control is given by 

u* = -2{x + c), 

where the constant c is defined in (14), in which the "— " sign is chosen and 
the constant Kq satisfies the condition (17). Furthermore, the value function 
is 

F{x) = l(x 3 - 4) + c{x 2 - d\) + (c 2 - \){x - di) for d x < x < d 2 . 

(b) Next, we consider the controlled stochastic process X(t) defined by 
the stochastic differential equation 

dX{t) = X{t) dt + {X^t^uit)]} 1 / 2 dW{t) 

and we look for the control u* that minimizes the expected value of the cost 
criterion 

J( x ) = f T{X \X{t)u{t) dt + K{X[t(x)}}. 
Jo 

That is, we take f[X(t)] = X(t), v[X(t)] = X 2 (t) and q[X(t)] = X(t). More- 
over, we set A = 0. In the case when u(t) = 1, X(t) is a geometric Brownian 
motion. Notice that 

v 2 (x) x 2 

~ ^=9{x). 



8q(x)f(x) 



It follows that the value function is given by [see (12) with n (= m) = 2, 
f = 1 and ho (= g ) = |] 

F(x) = 2V2cx(lnx - 1) + 2x(ln 2 x - 21nx + 2) + ax + c 

and 

G(x) = F"(x) = 2V2c- + 4—. 

X X 

Furthermore, the optimal control u* is 
(18) u* = - v / 2c-21nx. 

Next, (13) yields that 

c 2 + 21n 2 x + 2\/2clnx = 2v / 2clnx + 21n 2 x + ci c 2 = ci, 

which is in fact true for all problems in Case 2. Hence, we have 

F(x) = 2V2cx(lnx - 1) + 2xlnx(lnx - 2) + 4x + c 2 x + c . 



A HOMING PROBLEM 9 

The boundary conditions F(dx) = and F(d 2 ) = Kq imply that 

F(x) =2V2c[(x\nx-d l hxdi) ~ - di)] + 2(xln 2 x - d x ln 2 di) 

(19) 

- 4(xln x - di lndi) + 4(x - di) + c 2 (x - dx), 
where the constant c is such that 

K = c 2 (d 2 -di) + 2V2c[(d 2 lnd 2 - dx lndi) - (d 2 - dx)] 

+ 2d 2 In 2 d 2 - 2di In 2 d\ - 4d 2 \nd 2 + Ad\ lndi + 4(^2 - dx). 
For simplicity, let c?i = 1 and d 2 = d. Then, we have 
= (d- l)c 2 + 2V2c[dlnd- (d- 1)] + 2dln 2 d - 4dlnd + 4{d - 1) - K , 
so that 

_ -2y/2(dlnd - d + 1) ± [8dln 2 d - 8{d - l) 2 + 4(d - 1)K ] 1/2 

(20) c " 2{d^T) • 

Thus, a first restriction on the constant Kq is that 

K > 2(d - 1) - 2—^— In 2 d, 
d — 1 

where d > 1 . 

Finally, we know that we must also have 

F"(x) =2\/2c- + 4— < VxG[l,4 

x x 

which implies that 

c< -V2\nd. 

We then deduce from (20) that we must again choose the "— " sign and that 
the constant Kq must satisfy the inequality 

Ko >4(d- 1) -41nd-21n 2 d. 

Since 

4(d- 1) -41nd-21n 2 d> 2(d- 1) - 2-^^ln 2 d Vd> 1, 
we must impose the constraint 

(21) ET >4(d-l-lnd-iln 2 d). 

In summary, when we choose f[X(t)] =X(t), v[X(t)] = X 2 (t), q[X(t)) = 
X(t), A = 0, dx = 1 and d 2 = d (> 1), the optimal control is given by (18) 
with the constant c defined in (20), in which the "— " sign is chosen and the 
constant Kq is such that (21) holds. Moreover, the value function is [see (19) 
with dx = 1 and d 2 = d] 

F(x) = 2xlnx[lnx + V2c- 2] + (x - l)[c 2 - 2^2,0 + A). 



10 



M. LEFEBVRE 



4. Concluding remarks. In this article, the problem of optimally con- 
trolling a certain class of one-dimensional diffusion processes was set up and 
solved exactly. Contrary to the classic formulation, the diffusion processes in 
question had control-dependent infinitesimal variances rather than infinites- 
imal means. The objective, when the parameter A in the cost criterion J(x) 
defined in (3) is nonnegative, was to incite the controlled process to exit, as 
soon as possible, the interval [di,^] at d\. When A is negative, a reward is 
given for survival in the interval (di,^)- 

In Section 3, two particular cases were treated extensively. The first ex- 
ample involved a controlled Wiener process, whereas in the second example 
a geometric Brownian motion was optimally controlled. Many other im- 
portant cases could be considered. For instance, particular Bessel processes 
could have been used. It would also be interesting to take f[X(t)] = —aX(t), 
with a a positive constant, i>[X(t)] = vq > and A < 0. Then, the controlled 
process X(t) with u(t) = 1 is an Ornstein-Uhlenbeck process. Since / [A"(t)] 
is negative when X{t) is in the interval [di,^], we have J(x) = if A = 
and if the optimizer chooses u{t) = 0. However, if A is negative and small 
enough (i.e., large enough in absolute value), the optimizer can receive a re- 
ward overall if the controlled process remains between d\ and d,2 for a long 
enough time. Therefore, the optimal control u* will not always be identical 
to zero if A is small and Kq is not too large. Actually, the situation is similar 
even when f[X(t)] > in the interval [di,^]. Indeed, if Kq is not large, the 
optimizer is better off to let X(t) hit c?2 before d\ rather than to use a lot of 
control to make X(t) hit d\ first; hence the constraints that we must impose 
on the constant Kq, as we have seen in the examples presented in Section 3. 

Finally, possible extensions of the work presented in this article are the 
following: first, a two-dimensional version of the optimal control problem 
could be considered. Also, we could use a risk-sensitive cost criterion rather 
than J{x). 
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